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Status  of  Effort: 


This  contract  represents  a  one  year  increment  to  the  three  year  contract 
F49620-01-1-0013. 


In  May  2004,  Exxon-Mobil  sponsored  the  third  international  workshop  on 
surrogate  optimization.  This  workshop  was  held  at  Rice,  and  it  followed 
the  first  workshop  held  at  the  Danish  Technical  University  and  the  second 
held  in  Coimbra,  Portugal.  The  Rice  workshop  had  participation  from  several 
industrial  groups,  most  notably  Boeing  and  ExxonMobil,  as  well  as  SANDIA 
and  Lawrence  Berkeley  Labs.  Our  former  joint  PhD  student  and  present 
collaborator,  Lt  Col  Mark  Abramson  of  AFIT,  attended  as  well. 


The  papers  from  the  Danish  workshop  appeared  in  a  special  issue  of  BIT. 
The  papers  from  the  Portugal  workshop  have  just  appeared  in  a  special  issue 
of  Optimization  in  Engineering  edited  by  Audet,  Dennis,  and  Professor  Luis 
Vicente  of  Coimbra,  who  received  his  PhD  with  Dennis  at  Rice. 


The  statement  of  work  for  this  add-on  contract  was  mostly  concerned  with 
work  on  combining  our  new  MADS  algorithm  with  the  filter  method  for 
handling  constraints.  This  has  turned  out  to  be  a  difficult  problem,  but  we 
are  in  the  process  of  preparing  a  paper  now  on  the  subject. 


We  have  recently  made  an  important  breakthrough  in  both  the  theory  and 
the  practice  of  derivative-free  methods.  Our  new  research  eliminates  the 
main  limitation  of  the  Generalized  Pattern  Search  (GPS)  class  of  algorithms, 
which  formed  the  algorithmic  infrastructure  for  our  past  work.  Our  new 
work  addressed  the  issue  that  the  GPS  algorithm  class  relies  heavily  on  a 
fixed  finite  set  of  directions,  and  that  the  convergence  results  are  heavily 
tied  to  these  directions.  We  found  a  way  to  lift  this  restriction,  and  now  we 
allow  the  algorithm  to  explore  in  a  set  of  directions  which  is  dense  in  the 
entire  space.  We  call  this  the  Mesh  Adaptive  Direct  Search  (MADS)  class  of 
algorithm,  and  it  generalizes  GPS.  We  observed  significant  improvements  on 
some  interesting  problems. 


Lt  Col.  Abramson,  Audet,  and  Dennis  are  working  to  draft  a  graduate 
level  engineering  and  applied  mathematics  textbook  based  on  our  research. 
In  addition,  Lt  Col  Abramson  is  showing  himself  to  be  a  valuable  and  ac¬ 
tive  independent  researcher.  He  is  our  best  transition  to  the  USAF.  His 
dissertation  is  available  from  www.caam.rice.edu  as  TR02-11,  and  several 
subsequent  papers  can  be  found  there  as  well.  His  NOMADm  Matlab  imple¬ 
mentation  of  the  class  of  algorithms  we  study  was  acknowledged  to  be  the 
model  for  the  new  Matlab  toolbox  on  direct  search  and  genetic  algorithms, 
http:  // www.  mathworks.  com  /  products /gads /description4 .  j  sp 


Two  students  completed  the  courses,  and  passed  their  written  B-exams  for 
the  Ph.D.  in  Montreal  under  the  supervision  of  Audet.  Both  students  work 
on  extensions  of  MADS:  Sebastien  Le  Digabel’s  will  use  parallelism  and  re¬ 
duction  techniques  to  allow  GPS  to  tackle  larger  problems,  and  Walid  Zghal’s 
work  will  focus  on  multiobjective  and  bilevel  black  box  optimization.  Audet 
also  supervises  a  M.Sc.  student  that  specializes  MADS  to  a  process  opti¬ 
mization  problem  from  chemical  engineering. 


John  Dennis  served  on  the  PhD  committee  of  Parvez  Moin’s  Stanford  grad¬ 
uate  student  Alison  Marsden,  whose  dissertation  deals  with  applying  the 
surrogate  management  framework  to  obtain  quiet  trailing  edge  airfoil  de¬ 
signs.  We  have  published  two  joint  papers  on  this  work,  and  Ms  Marsden  is 
presently  applying  MADS  to  fully  turbulent  flow  governed  by  a  large  eddy 
simulation.  The  optimization  has  been  running  for  six  months,  and  it  has 
achieved  an  80%  decrease  in  acoustic  noise  so  far.  Dr.  Marsden  is  presently  a 
postdoc  in  the  Stanford  Medical  School,  where  her  task  is  to  apply  our  ideas 
to  bioengineering  design.  We  continue  to  work  together. 


Accomplishments/New  Findings: 


Especially  noteworthy  accomplishments  are: 


•  Several  papers  have  appeared  or  been  accepted  by  top  peer  reviewed 
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journals.  Our  papers  combine  theoretical  results,  analysis  of  the  limita¬ 
tions  of  GPS  and  numerical  results  on  some  real  engineering  problems. 

•  The  new  MADS  framework  will  replace  our  GPS  framework.  MADS 
is  much  more  flexible,  and  allows  us  to  show  the  convergence  results 
that  we  always  wished  for  in  GPS,  but  knew  did  not  hold.  This  opens 
the  door  for  updating  all  our  previous  papers  into  this  new  framework. 
This  will  be  done  in  the  book. 

•  Dennis  has  taken  the  lead  in  developing  with  Boeing  researchers  a  more 
mathematical  approach  to  the  area  called  systems  of  systems.  This  area 
has  many  military  and  civilian  applications.  Most  of  the  papers  in  the 
literature  are  written  from  a  systems  engineering  approach,  which  Boe¬ 
ing  feels  is  not  sufficient  to  establish  a  unified  approach  to  algorithms 
for  such  problems.  Copies  of  these  preprints  will  be  forwarded  sepa¬ 
rately  to  Dr.  Glassman. 


Personnel  Supported: 


Faculty:  Professor  John  Dennis 


Publications: 


Published: 


•  Mark  A.  Abramson,  Charles  Audet,  and  J.  E.  Dennis,  Jr.  Generalized 
pattern  searches  with  derivative  information.  Mathematical  Program¬ 
ming,  Series  B,  100:3-25,  2004. 

•  Charles  Audet  and  J.  E.  Dennis,  Jr.  Analysis  of  generalized  pattern 
searches.  SIAM  Journal  on  Optimization,  13(3):889— 903,  2003. 
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•  Charles  Audet  and  J.  E.  Dennis,  Jr.  A  pattern  search  filter  method 
for  nonlinear  programming  without  derivatives.  SIAM  Journal  on 
Optimization ,  14(4):980-1010,  2004. 

•  J.  E.  Dennis,  Jr.,  C.  Price  and  I.  Coope)  Direct  Search  Methods  for 
Nonlinearly  Constrained  Optimization  Using  Filters  and  Frames,  Op¬ 
timization  and  Engineering  5(2),  pp  123-144,  2004. 

•  Alison  Marsden,  Meng  Wang,  J.  E.  Dennis,  Jr.,  and  Parviz  Moin,  Opti¬ 
mal  aeroacoustic  shape  design  using  the  surrogate  management  frame¬ 
work,  Optimization  and  Engineering ,  5(2):  101-122,  2004. 

•  Alison  L.  Marsden,  Meng  Wang,  J.  E.  Dennis,  Jr.,  and  Parviz  Moin), 
Suppression  of  vortex-shedding  noise  via  derivative-free  shape  opti¬ 
mization  (Physics  of  Fluids),  16(10)  2004,  p.  L83-L86  2004. 


Accepted  for  Publication: 
None  at  present 
Submitted  for  Publication: 


•  Mark  A.  Abramson,  Charles  Audet,  and  J.  E.  Dennis,  Jr.  Filter 
pattern  search  algorithms  for  mixed  variable  constrained  optimization 
problems.  Technical  Report  TR04-09,  CAAM  technical  report,  Rice 
University,  2004.  submitted  to  Pacific  Journal  of  Optimization. 

•  Mark  A.  Abramson,  Olga  A.  Brezhneva,  and  J.  E.  Dennis,  Jr.  Pattern 
Search  Methods  for  Linearly  Constrained  Minimization  in  the  Presence 
of  Degeneracy,  IMA  Preprint  series,  #  1934,  and  Rice  CAAM-TR03-09, 
August  2003. 

•  Charles  Audet  and  J.  E.  Dennis,  Jr.  Mesh  adaptive  direct  search 
algorithms  for  constrained  optimization.  Technical  Report  G  -2004- 
04,  Les  Cahiers  du  GERAD,  2004.  submitted  to  SIAM  JOPT. 
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Interactions  and  Transitions: 


Public  Presentations: 


J.  E.  Dennis 


•  “Optimization  using  Surrogates  for  Engineering  Design”.  Departmen¬ 
tal  Colloquium,  Mechanical  Engineering  Department,  University  of 
Michigan,  Ann  Arbor,  April  16,  2004. 

•  “Optimization  using  Surrogates  for  Engineering  Design” .  RODD  AFOSR 
Meeting,  Shalimar  Florida,  April  20,  2004. 

•  “Mesh  Adaptive  Direct  Search  Algorithms” .  AFOSR  Program  Review, 
Pasadina,  California,  August  11,  2004. 

•  “Mesh  Adaptive  Direct  Search  Algorithms”.  Optimization  seminar, 
University  of  Wisconsin,  Madison,  September  23,  2004. 

•  “Optimization  using  Surrogates  for  Engineering  Design” .  Departmen¬ 
tal  Colloquium,  Industrial  Engineering  Department,  University  of  Wis¬ 
consin,  Madison,  September  24,  2004. 

•  ’’Optimization  Using  Surrogates  for  Engineering  Design.”  Boeing  Dis¬ 
tinguished  Scholar  Series,  Webcast  company  wide,  December  2,  2004. 

•  “Optimization  using  Surrogates  for  Engineering  Design”.  Industrial 
Mathematics  Series,  Fields  Institute  for  Mathematics,  University  of 
Toronto,  February  1,  2004 

•  “Mesh  Adaptive  Direct  Search  Algorithms”.  Optimization  seminar, 
MacMaster  University,  Hamilton,  Ontario,  February  3,  2004. 

•  “Mesh  Adaptive  Direct  Search  Algorithms”.  Optimization  seminar, 
ExxonMobil  Upstream  Research,  Houston,  February  10,  2005. 

•  “Mathematical  Formulations  of  Systems  of  Systems  Problems” .  Exxon¬ 
Mobil  Upstream  Research,  Houston,  February  11,  2005. 
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•  “Optimization  using  Surrogates  for  Engineering  Design”.  West  Coast 
Optimization  Meeting,  University  of  Washington,  Seattle,  April  30, 
2005. 


Editorial  positions: 

J.  E.  Dennis 

2001-  Editorial  Board,  SIAM  Journal  on  Optimization 

1992-  Advisory  Editor,  Mathematics  of  Operations  Research 

2003-2004  Invited  Editor  for  Optimization  and  Engineering  (spe¬ 
cial  issue  on  Surrogate  Optimization  ) 

Consultative  and  Advisory  Functions: 

J.  E.  Dennis 

•  Renewal  site  visit  team  for  NSF  Institute  for  Pure  and  Applied  Math¬ 
ematics  at  UCLA,  October  2004. 

•  One  day  per  week  at  Boeing  Phantom  Works  Mathematics  and  Engi¬ 
neering  Analysis  Group  working  on  Design  Explorer  Project  and  Sys¬ 
tems  of  Systems  optimization. 

•  Advisory  Committee  for  Worcester  Polytechnic  Institute. 

•  ExxonMobil  Upstream  Research  Company 

•  United  Technologies  Research  Center 

•  Lawrence  Berkeley  Laboratory  Nanotechnology  project. 
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•  Served  on  PhD  committee  for  Stanford  University  Mechanical  Engi¬ 
neering  doctoral  candidate  Alison  Marsden. 


Transitions: 


Projects  in  the  Exploratory  Stage 

•  Systems  of  Systems  With  a  group  of  Boeing  collaborators,  we  are  look¬ 
ing  at  systems  of  systems  problems  from  a  mathematical  point  of  view. 
Previous  work  has  come  mostly  from  the  military  operations  research 
community,  and  it  has  been  largely  concerned  with  ad  hoc  solutions  of 
specific  cases.  We  have  written  a  Boeing  Technical  Report  (nonpropri¬ 
etary),  which  has  been  invited  for  presentation  at  the  upcoming  IEEE 
meeting  on  Systems  of  Systems.  We  are  almost  finished  with  another 
more  general  paper.  Our  approach  is  to  categorize  SoS  problems  de¬ 
pending  on  the  degree  of  control  exercised  by  the  “central  authority” 
ranging  from  inactive  to  omnipotent. 

Contact:  Dr.  Evin  Cramer  (206)232-9577  or 
Dr.  Greg  Shubin  (425)865-3516. 

•  Well  casing  All  of  the  personnel  who  received  support  from  this  grant 
spent  some  time  at  ExxonMobil  Upstream  Research  Company  talking 
about  various  collaborations.  One  problem  presently  being  discussed 
is  an  interesting  mixed  integer  problem,  which  is  described  briefly  as 
follows.  As  a  well  is  being  drilled,  it  is  important  to  insert  sleeves  of 
well  casing  to  keep  the  sides  of  the  hole  from  collapsing  into  the  well. 
The  first  sleeve  is  the  largest  in  diameter,  the  second  is  inserted  inside 
the  first  and  lowered  into  place,  and  this  concentric  sleeving  continues 
until  the  drilling  is  completed  and  the  wall  of  the  well  is  completely 
reinforced  by  concentric  sleeves  of  decreasing  diameter.  The  problem 
is  to  determine  the  number  of  sleeves,  a  categorical  variable,  and  the 
length  and  diameter  of  each  sleeve  for  a  well  of  given  depth.  Of  course, 
there  are  various  structural  constraints,  and  cost  is  the  objective  to  be 
minimized.  We  should  be  able  to  deal  with  this  problem  with  little 


8 


difficulty,  but  the  delay  is  due  to  a  need  to  obtain  internal  funding  for 
ExxonMobil  personnel  to  gather  all  the  necessary  simulation  routines. 
The  status  of  this  project  is  that  NOMAD  has  been  successfully  used 
by  ExxonMobil  to  solve  the  problem  for  a  single  section. 

Contact:  Dr.  Amr  ElBakry  (713)431-7137. 

•  Nozzle  Design  Our  group  continues  working  with  a  group  in  Boeing 
Commercial  Airplane  Group  (BCAG)  to  reduce  the  cycle  time  for  de¬ 
signing  nozzles.  A  nozzle  is  the  inside  part  of  the  engine  housing,  and 
its  design  is  affected  by  the  design  of  the  more  “upstream”  components. 
This  means  that  many  of  the  design  changes  involving  other  airplane 
components  force  a  redesign  of  the  nozzle  as  well. 

A  nozzle  design  is  specified  by  100  parameters,  90  of  which  are  fixed  by 
other  considerations.  The  system  is  governed  by  a  2D  Navier  Stokes 
coupled  with  NASTRAN,  a  commercial  structures  code.  It  takes  about 
3  hours  on  an  SGI  Challenge  to  get  one  function  value.  The  current 
length  of  a  design  cycle  is  two  weeks.  We  expect  to  reduce  that  to 
approximately  one  day. 

This  project  has  been  delayed  by  the  higher  internal  Boeing  priority  of 
the  planform  project. 

Contact:  Dr.  Evin  Cramer  (206)232-9577  or 
Dr.  Greg  Shubin  (425)865-3516. 


Ongoing  Projects 

•  Model  Management  Framework  Software  Boeing  has  the  lat¬ 
est  NOMAD-C++  release  as  well  as  the  NOMADm  MatLab  software. 
When  internal  funding  permits,  this  will  likely  become  the  basis  for 
reimplementation  of  their  Design  Explorer  Software.  This  implementa¬ 
tion  replaces  the  C++  box  constrained  prototype  delivered  previously. 
The  framework  can  be  used  to  develop  model  management  algorithms 
specialized  for  a  user’s  applications  and  capabilities.  The  Boeing  ver¬ 
sion  of  our  work,  Design  Explorer,  is  presently  being  commercialized 
by  Phoenix  Integration. 
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Contact:  Dr.  Evin  Cramer  (206)232-9577  or 
Dr.  Greg  Shubin  (425)865-3516. 

•  Planform  Design  Our  algorithms  were  used  in  a  production  environ¬ 
ment  by  Boeing  in  planform  design  for  the  near  sonic  cruiser.  This  was 
done  by  the  BCAG  group  that  does  planform  design.  The  decision  was 
made  not  to  build  this  airplane  because  of  economic  conditions  in  the 
airline  industry.  Now,  Design  Explorer  is  being  used  on  the  proposed 
Dreamliner  7e7.  Boeing  continues  to  achieve  better  planforms  with 
greatly  reduced  cycle  time. 

The  planform  is  the  shape  of  the  wing  as  viewed  from  above.  It  is 
a  tricky  design  problem  that  involves  a  couple  of  dozen  variables  and 
some  interesting  constraints.  For  example,  the  fuel  is  outboard  on 
the  wings,  and  the  wings  must  be  swept  back  for  performance.  One 
constraint  is  that  the  plane  must  not  fall  over  backwards  when  the 
tanks  are  filled  while  the  plane  is  on  the  ground.  This  problem  involves 
multiple  objectives  and  discrete  variables,  but  the  first  test  problem  run 
by  Boeing  used  cost/passenger  mile  as  objective  and  assumed  values 
for  the  discrete  variables,  like  where  the  wings  are  placed  on  the  body. 

The  following  text  on  the  recent  test  was  approved  by  Boeing  for  release: 

Recent  Product  Development  activities  at  Boeing  have  underscored  the 
need  to  decrease  the  time  to  market  and  cost  of  products.  To  accom¬ 
plish  this,  Boeing  has  been  examining  all  of  its  processes  in  development 
and  production. 

One  process  in  particular  that  occurs  early  in  the  design  is  the  determi¬ 
nation  of  a  wing  planform,  which  must  be  chosen  to  meet  mission  while 
decreasing  operating  economics  and  respecting  design  constraints.  Boe¬ 
ing  is  exploring  whether  this  highly  multi-disciplinary  problem  yields 
itself  to  integrated  analysis  and  optimization  techniques,  with  the  hope 
of  decreasing  the  cycle  time  of  finding  a  preferred  result  by  an  order  of 
magnitude. 

This  exploration  provided  the  Design  Explorer  Team,  of  which  Rice  is 
a  member,  with  an  opportunity  to  apply  our  design  optimization  tools. 
It  has  also  been  useful  from  the  standpoint  of  exercising  the  various 
discipline  analysis  tools.  Starting  from  baselines  developed  through 
the  standard  process  and  using  a  significant  subset  of  the  tools,  we 
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have  been  able  to  demonstrate  actual  integration  of  the  analyses  from 
the  different  disciplines  and  the  speed  advantages  that  we  were  looking 
for.  The  solutions  found  do,  within  the  level  of  accuracy  of  the  included 
tools,  meet  all  constraints  while  showing  economic  benefit  relative  to 
our  analyses  of  the  initial  baselines. 

The  approach  to  this  problem  used  response  surface  (surrogate)  mod¬ 
eling  to  speed  up  the  optimization,  as  opposed  to  continuously  re¬ 
sampling  the  analysis  tools.  Of  additional  note  is  that  incorporation  of 
Model  Management  Framework  elements  into  the  optimization  scheme 
allowed  resolution  of  the  selected  optimum  to  search  variables  of  sec¬ 
ond  order  importance.  Such  resolution  is  particularly  difficult  to  obtain 
when  using  response  surface  modeling. 

Contact:  Dr.  Evin  Cramer  (206)232-9577  or 
Dr.  Greg  Shubin  (425)865-3516. 

•  Boeing  Parts  Nesting  System  PDS  continues  to  be  used  in  the 
Boeing  Parts  Nesting  System  2NA  for  just-in-time  manufacturing  of 
aircraft  parts.  This  code  runs  over  100,000  times  per  day. 

•  Optimal  algorithmic  parameters  We  devised  a  general  framework 
for  parameter  tuning  that  uses  MADS  to  identify  good  algorithmic 
parameters.  The  framework  makes  provision  for  surrogate  objectives. 
Parameters  are  sought  so  as  to  minimize  some  measure  of  performance 
of  the  algorithm  being  fine-tuned.  This  measure  is  treated  as  a  black¬ 
box  and  may  be  chosen  by  the  user. 

We  specialized  this  framework  to  the  identification  of  trust-region  pa¬ 
rameters  in  unconstrained  optimization.  Parameters  are  identified  that 
minimize,  in  a  certain  sense,  the  computational  time  or  the  number  of 
function  evaluations  required  to  solve  a  set  of  problems  from  the  CUTER 
collection.  Each  function  call  may  take  several  hours  and  may  not  al¬ 
ways  return  a  predictable  result.  A  surrogate  function,  tailored  to  the 
experiment  at  hand,  is  used  to  guide  MADS  towards  a  local  solution. 

Contact:  Dr.  Dominique  Orban  (514)340-4711  ext  5967 

•  Trailing  edge  design  We  have  a  collaboration  with  Parviz  Moin’s  tur¬ 
bulent  flow  group  in  the  Mechanical  Engineering  Department  at  Stan¬ 
ford.  Our  algorithms  have  been  successful  in  achieving  airfoil  designs 


11 


with  over  50%  reduction  in  noise  from  flow  over  the  wings.  There  are 
several  important  application  of  this  work  from  windmills  to  aircraft. 

Contact:  Dr.  Alison  Marsden  (650)  723-2416 


Inventions,  or  Patent  Disclosures: 


None. 


Honors  /Awards: 


J.  E.  Dennis 

•  Very  Important  Visitor  at  Institute  for  Mathematics  and  its  Applica¬ 
tions  at  the  University  of  Minnesota  Winter  2003 

•  Ordway  Distinguished  Lecturer  in  the  Mathematics  Department  at  the 
University  of  Minnesota  Winter  2003 

•  Chair  of  the  organizing  committee  for  the  SIAM  2004  Annual  Meeting, 

August,  Portland,  OR 

•  Named  a  HighlyCited  Computer  Scientist  by  ISI. 

•  Dedicatee  of  special  issue  of  the  SIAM  Journal  on  Optimization  (Vol¬ 
ume  9,  Number  4)  Chttp :  / /epubs .  siam .  org/sam-bin/ dbq/toc/SIOPT/9/ 4>. 

•  Chair(President)  of  the  Mathematical  Programming  Society 

•  Founder  and  Editor-in-Chief  of  MPS/SIAM  Book  Series  on  Optimiza¬ 
tion 

•  Advisory  Editor,  Mathematics  of  Operations  Research,  (1992 — ). 
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•  Founding  Editor-In-Chief  of  SIAM  Journal  on  Optimization. 

•  Served  on  and  chaired  numerous  panels  and  visiting  committees. 

•  Served  two  terms  on  the  SIAM  Council. 

•  Fulbright  Lecturer  to  Argentina. 

•  Three  times  an  Erskine  Fellow  at  the  University  of  Canterbury,  Christchurch, 
NZ,  2000,  2002,  and  2004. 
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